User-driven Online Kernel Fusion for SYCL

Heterogeneous programming models are becoming increasingly popular to support the ever-evolving hardware architectures, especially for new and emerging specialized accelerators optimizing specific tasks. While such programs provide performance portability of existing applications across various heterogeneous architectures some extent, short-running device kernels can affect an application due overheads data transfer, synchronization, kernel launch. in with one or two overhead be negligible, it noticeable when these dominate overall number application, as is case graph-based neural network models, where there several small memory-bound nodes alongside few large compute-bound nodes. To reduce overhead, combining into a single, more optimized active area research. However, this task time-consuming error-prone given huge set potential combinations. This push programmers seek tradeoff between (a) task-specific low but hard maintain (b) smaller modular higher easier maintain. DSL-based approaches, those provided machine learning frameworks, which offer possibility fusion, they limited particular domain exploit knowledge that and, consequence, port elsewhere. study explores feasibility user-driven fusion through extension SYCL API address automation fusion. The proposed solution requires define subgraph regions potentially suitable without any modification code function signature. We evaluate benefit our approach on common networks improvement detail.